DATA GENERATION

Reinforced Counterfactual Data Augmentation for Dual Sentiment Classification (2021.emnlp-main)

Challenge:	Existing methods for low-resource relation extraction (LRE) lack diversity, leading to suboptimal performance.
Approach:	They propose to use large language models to augment relation extraction models by observing the RE model's behavior and replacing schema constraints with attribute constraints.
Outcome:	Experiments on three widely-used benchmarks show that the proposed method outperforms state-of-the-art methods while maintaining enhanced model stability.

Copied to clipboard

Hao Chen, Rui Xia, Jianfei Yu

Information Retrieval

Challenge:	Existing approaches to improve generalization ability by augmenting training data with synonymous examples or adding random noises to word embeddings cannot address spurious association problem.
Approach:	They propose an end-to-end reinforcement learning framework which jointly performs counterfactual data generation and dual sentiment classification.
Outcome:	The proposed framework outperforms strong data augmentation baselines on several benchmark sentiment classification datasets.

Proofread: Fixes All Errors with One Tap (2024.acl-demos)

Copied to clipboard

Renjie Liu, Yanxiang Zhang, Yun Zhu, Haicheng Sun, Yuanbo Zhang, Michael Huang, Shanqing Cai, Lei Meng, Shumin Zhai

On Synthetic Data for Back Translation (2022.naacl-main)

Challenge:	Extensive experiments on a human-labeled golden set showed our tuned PaLM2-XS model achieved 85.56% good ratio.
Approach:	They propose a two-stage tuning approach to acquire the dedicated Large Language Model for the feature, followed by a reinforcement learning approach for targeted refinement.
Outcome:	The proposed model achieves 85.56% good quality on Rewrite and proofread tasks on human-labeled golden sets.

Copied to clipboard

Jiahao Xu, Yubin Ruan, Wei Bi, Guoping Huang, Shuming Shi, Lihui Chen, Lemao Liu

Text Generation

Challenge:	Existing studies on back translation (BT) focus on beam search or random sampling . a new method to generate synthetic data with a backward model is proposed to improve BT performance.
Approach:	They propose a method to generate synthetic data to trade off quality and importance factors . back translation (BT) is one of the most significant technologies in NMT research fields .
Outcome:	The proposed method outperforms the baseline methods on WMT14 DE-EN, EN-DE, and RU-EN benchmark tasks.

LATEX-Numeric: Language Agnostic Text Attribute Extraction for Numeric Attributes (2021.naacl-industry)

Copied to clipboard

Kartik Mehta, Ioana Oprea, Nikhil Rasiwasia

Low-Resource NLP

Challenge:	Existing methods for training numeric attributes are based on manual labeling and distant supervision leads to incomplete training annotations.
Approach:	They propose a multi-task learning architecture to deal with missing attribute values in training data, removing dependency on manual annotations.
Outcome:	The proposed framework improves on 20 numeric attributes extracted from 5 product categories and 3 english marketplaces with language-agnostic performance.

DecIF: Improving Instruction-Following through Decomposition (2026.acl-long)

Copied to clipboard

Tingfeng Hui, Pengyu Zhu, Bowen Ping, Ling Tang, Guanting Dong, Yaqi Zhang, Sen Su

Challenge:	Existing approaches to obtain high-quality instruction-following data rely heavily on existing documents and existing methods.
Approach:	They propose a data synthesis framework, DecIF, which automatically generates accurate and diverse instruction-following data from scratch for supervised fine-tuning and reinforcement learning.
Outcome:	Extensive experiments show that the proposed framework can synthesize accurate instruction-following data for both SFT and RL paradigms compared to baselines.

AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applications (2023.emnlp-industry)

Copied to clipboard

Bhaktipriya Radharapu, Kevin Robinson, Lora Aroyo, Preethi Lahoti

Robustness in NLP

Challenge:	Large Language Models (LLMs) are rapidly becoming more and more popular, but dealing with the potential harms associated with their deployment in real-world scenarios is still an open research question.
Approach:	They propose an automated approach for automated generation of adversarial evaluation datasets to test the safety of LLM generations on new downstream applications.
Outcome:	AART generates evaluation datasets with high diversity of content characteristics critical for effective adversarial testing.

Improving Sentence Embeddings with Automatic Generation of Training Data Using Few-shot Examples (2024.acl-srw)

Copied to clipboard

Soma Sato, Hayato Tsukagoshi, Ryohei Sasano, Koichi Takeda

Learning to Rewrite Negation Queries in Product Search (2025.coling-industry)

Challenge:	Decoder-based large language models (LLMs) have shown high performance on many tasks in natural language processing.
Approach:	They propose to automatically generate an NLI dataset with an LLM and use it for fine-tuning of PromptEOL.
Outcome:	The proposed model outperforms existing models on STS tasks without large manually annotated datasets.

Copied to clipboard

Mengtian Guo, Mutasem Al-Darabsah, Choon Hui Teo, Jonathan May, Tarun Agarwal, Rahul Bhagat

Text Generation

Challenge:	Negations in product search are often used to articulate unwanted product features or components.
Approach:	They propose a query rewriting approach to enhance product search performance . they use large language models to extract query reawrites from product text . their results pave the way for further research on enhancing search performance of queries with negations .
Outcome:	The proposed approach improves search performance by 3.17% for queries with negations.

Generating Complex Question Decompositions in the Face of Distribution Shifts (2025.naacl-long)

Copied to clipboard

Kelvin Han, Claire Gardent

Natural Language Interfaces

Challenge:	Question decomposition has been found to improve large language models’ (LLMs) performance on complex question answering (QA) however, performance on the task remains dominated by supervised approaches, suggesting room for making LLMs better decomposers.
Approach:	They propose to generate synthetic decomposition data with only five annotated examples by extending recent advances in using LLM-as-judge and for reranking in novel ways.
Outcome:	The proposed approach generates synthetic decomposition data with only five examples over two benchmark datasets.

DEAM: Dialogue Coherence Evaluation using AMR-based Semantic Manipulations (2022.acl-long)

Copied to clipboard

Sarik Ghazarian, Nuan Wen, Aram Galstyan, Nanyun Peng

Dialogue Systems & Conversational Agents

Challenge:	Existing evaluation metrics for incoherent dialogues are insufficient to accurately reflect incohérence . despite the effectiveness of large pretrained language models, not everyone is into this type of work.
Approach:	They propose a Dialogue coherence Evaluation metric that uses Abstract Meaning Representation to apply semantic-level Manipulations for incoherent (negative) data generation.
Outcome:	The proposed evaluation metric achieves higher correlations with human judgments compared to baseline methods on dialog datasets by significant margins.

AIDE: Attribute-Guided MultI-Hop Data Expansion for Data Scarcity in Task-Specific Fine-tuning (2025.acl-industry)

Copied to clipboard

Jiayu Li, Jennifer Zhu, Fang Liu, Yanjun Qi

Solver-Independent Automated Problem Formulation via LLMs for High-Cost Simulation-Driven Design (2026.findings-acl)

Challenge:	Existing methods for fine-tuning large language models for specific tasks require extensive seed datasets or struggle to balance task relevance and data diversity.
Approach:	They propose a data synthesis framework that uses a multi-hop process to expand very few seed data points while ensuring data diversity and task relevance.
Outcome:	The proposed framework outperforms state-of-the-art methods in task-specific fine-tuning by over 30%.

Copied to clipboard

Yuchen Li, Handing Wang, Bing Xue, Mengjie Zhang, Yaochu Jin

Efficient Data Generation for Source-grounded Information-seeking Dialogs: A Use Case for Meeting Transcripts (2024.findings-emnlp)

Challenge:	Existing approaches to translating ambiguous design requirements into a mathematical optimization formulation are expensive and time-consuming.
Approach:	They propose a solver-independent framework that converts engineers’ natural language requirements into executable optimization models.
Outcome:	The proposed framework outperforms existing methods in the accuracy of requirement formalization and quality of resulting radiation efficiency curves on antenna design.

Copied to clipboard

Lotem Golany, Filippo Galgani, Maya Mamo, Nimrod Parasol, Omer Vandsburger, Nadav Bar, Ido Dagan

Dialogue Systems & Conversational Agents

Challenge:	Existing methods for automating data generation with Large Language Models (LLMs) are difficult, and we propose a semi-automatic approach to generate dialogs with attributions.
Approach:	They propose a semi-automatic approach to generate dialog queries and responses with Large Language Models followed by human verification and identification of attribution spans.
Outcome:	The proposed approach improves the quality of the response generation and attribution quality of MISeD datasets while reducing time and effort.

OmniCompliance-100K: A Multi-Domain, Rule-Grounded, Real-World Safety Compliance Dataset (2026.findings-acl)

Copied to clipboard

Wenbin Hu, Huihao Jing, Haochen Shi, Changxuan Fan, Haoran Li, Yangqiu Song

Learning Disentangled Semantic Spaces of Explanations via Invertible Neural Networks (2024.acl-long)

Challenge:	Existing LLM safety datasets rely on ad-hoc taxonomies and lack rule-grounded, real-world cases.
Approach:	They construct a rule-grounded, real-world case dataset OmniCompliance-100K from a compliance perspective using a powerful web-searching agent.
Outcome:	The proposed dataset spans 74 regulations and policies across a wide range of domains including security and privacy regulations, content safety and user data privacy policies, financial security requirements, medical device risk management standards, educational integrity guidelines, and protections of fundamental human rights.

Copied to clipboard

Yingji Zhang, Danilo Carvalho, Andre Freitas

Explainability & Interpretability in NLP

Challenge:	Disentangled latent spaces usually have better semantic separability and geometrical properties, which leads to better interpretability and controllable data generation.
Approach:	They propose a flow-based invertible neural network mechanism integrated with a transformer-based language Autoencoder to deliver latent spaces with better semantic separability and geometrical properties.
Outcome:	The proposed model can deliver latent spaces with better separability properties compared to the current state-of-the-art models.

MaXM: Towards Multilingual Visual Question Answering (2023.findings-emnlp)

Copied to clipboard

Soravit Changpinyo, Linting Xue, Michal Yarom, Ashish Thapliyal, Idan Szpektor, Julien Amelot, Xi Chen, Radu Soricut

Multimodality

Challenge:	Visual Question Answering (VQA) has been studied in the English language, but in other languages it would require a considerable amount of resources.
Approach:	They propose scalable solutions to multilingual visual question answering using an English language framework and an annotation protocol.
Outcome:	The proposed framework reduces human annotation efforts and creates a test-only VQA benchmark in 7 languages.

Performance-Guided LLM Knowledge Distillation for Efficient Text Classification at Scale (2024.emnlp-main)

Copied to clipboard

Flavio Palo, Prateek Singhi, Bilal Fadlallah

Information Retrieval

Challenge:	Large Language Models (LLMs) face high computational demands at inference time due to high computational costs.
Approach:	They propose a cost-effective and high-throughput solution for large language models . PGKD distills the knowledge of LLMs into smaller, task-specific models based on teacher-student knowledge distillation .
Outcome:	PGKD outperforms BERT-based models and other knowledge distillation methods on multi-class classification datasets.

Omni-Chart-600K: A Comprehensive Dataset of Chart Types for Chart Understanding (2025.findings-naacl)

Copied to clipboard

Shulei Wang, Shuai Yang, Wang Lin, Zirun Guo, Sihang Cai, Hai Huang, Ye Wang, Jingyuan Chen, Tao Jin

SFMSS: Service Flow aware Medical Scenario Simulation for Conversational Data Generation (2025.findings-naacl)

Challenge:	Existing chart-related training methods lack capabilities in information extraction, mathematical reasoning, and understanding of multiple chart types.
Approach:	They propose a two-stage training strategy and method for jointly training a vision encoder tailored for multi-type charts to address the deficiencies in chart types and limited scope of chart tasks in existing datasets.
Outcome:	The proposed dataset includes 21 diverse chart types and tasks, including data retrieval and mathematical reasoning.

Copied to clipboard

Zhijie Bao, Qingyun Liu, Xuanjing Huang, Zhongyu Wei

Natural Language Interfaces

Challenge:	Medical-specific Large Language Models (LLMs) have demonstrated impressive performance on medical-related exams and tasks.
Approach:	They propose a framework for medical conversational data generation that uses Authentic Seed Data to ensure quality of the data.
Outcome:	The proposed model outperforms all baselines and human evaluations, and aligns with human preferences and clinical demands.

GOLD: Generalized Knowledge Distillation via Out-of-Distribution-Guided Language Data Generation (2024.findings-naacl)

Copied to clipboard

Mohsen Gholami, Mohammad Akbari, Tianxi Hu, Vaden Masrani, Z. Wang, Yong Zhang

Corpora Generation for Grammatical Error Correction (N19-1)

Challenge:	Existing methods to generate data using LLMs are limited by sampling from the center of original content distribution.
Approach:	They propose a task-agnostic data generation and knowledge distillation framework for LLMs that employs an iterative out-of-distribution-guided feedback mechanism to generate data.
Outcome:	The proposed framework outperforms prior arts and the LLM on 10 different classification tasks and noisey generated data.

Copied to clipboard

Jared Lichtarge, Chris Alberti, Shankar Kumar, Noam Shazeer, Niki Parmar, Simon Tong

Machine Translation

Challenge:	Grammatical Error Correction (GEC) is a computational task that requires large amounts of data to solve.
Approach:	They propose two approaches to generate large parallel datasets for GEC using publicly available Wikipedia edit histories using minimal filtration heuristics and round-trip translation through bridge languages.
Outcome:	The proposed methods yield similar sized parallel corpora with around 4B tokens and are far ahead of the state-of-the-art on the CoNLL ‘14 benchmark and the JFLEG task.

EPiDA: An Easy Plug-in Data Augmentation Framework for High Performance Text Classification (2022.naacl-main)

Copied to clipboard

Minyi Zhao, Lu Zhang, Yi Xu, Jiandong Ding, Jihong Guan, Shuigeng Zhou

Logical DA: Enhancing Data Augmentation for Logical Reasoning via a Multi-Agent System (2025.findings-acl)

Challenge:	Existing methods for data augmentation do not fully exploit the potential of DA in NLP.
Approach:	They propose an easy and plug-in framework for data augmentation to support effective text classification.
Outcome:	The proposed framework outperforms existing methods in most cases, but not using agent networks or pre-trained generation networks.

Copied to clipboard

Haoqi Zheng, DongWang DongWang, Silin Yang, Yunpeng Qi, Ruochun Jin, Liyang Xu

Low-Resource NLP

Challenge:	Existing data augmentation paradigms isolate data synthesis from label validation, thereby reducing their utility for complex reasoning tasks.
Approach:	They propose a framework for enhancing reasoning-focused data augmentation in few-shot learning scenarios that integrates four agents through two synergistic phases: diverse data generation and label verification.
Outcome:	The proposed framework achieves the highest average improvement in task accuracy in both fine-tuning and in-context learning paradigms.

LLM-Forest: Ensemble Learning of LLMs with Graph-Augmented Prompts for Data Imputation (2025.findings-acl)

Copied to clipboard

Xinrui He, Yikun Ban, Jiaru Zou, Tianxin Wei, Curtiss Cook, Jingrui He

A Survey to Recent Progress Towards Understanding In-Context Learning (2025.findings-naacl)

Challenge:	Existing frameworks for missing data imputation are lacking in a finetuning-free process and mitigating biases and uncertainty in LLM outputs.
Approach:	They propose a framework for imputation of large language models with a forest of few-shot learning LLM "trees" they use bipartite information graphs to identify relevant neighboring entries with feature and value granularity.
Outcome:	The proposed framework is based on a concept of bipartite information graphs to identify high-quality relevant neighboring entries with both feature and value granularity.

Copied to clipboard

Haitao Mao, Guangliang Liu, Yao Ma, Rongrong Wang, Kristen Johnson, Jiliang Tang

LLM Distillation for Efficient Few-Shot Multiple Choice Question Answering (2025.findings-emnlp)

Challenge:	Existing research on In-Context Learning (ICL) is unclear, despite empirical success . a data generation perspective is used to interpret ICL .
Approach:	They propose to use data generation to reinterpret recent efforts from a systematic angle to demonstrate the potential broader usage of ICL.
Outcome:	The proposed model can learn from examples provided in the prompt, enabling downstream generalization without the need for gradient updates.

Copied to clipboard

Patrick Sutanto, Joan Santoso, Esther Irawati Setiawan, Aji Prasetya Wibawa

Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Models (2024.emnlp-main)

Challenge:	Large Language Models excel at few-shot learning but their direct application in real-world scenarios is often hindered by their high computational cost.
Approach:	They propose a framework that uses Large Language Models for data generation and scoring to improve encoder model performance.
Outcome:	The proposed approach improves accuracy from 28.9% to 39.3% on a few-shot MCQA task .

Copied to clipboard

Fei Wang, Ninareh Mehrabi, Palash Goyal, Rahul Gupta, Kai-Wei Chang, Aram Galstyan

TT-SI: Self-Improving LLM Agents with Test-Time Training (2026.findings-acl)

Challenge:	Recent studies have explored using LLMs for efficient data collection.
Approach:	They propose a method that takes into account the characteristics of the desired dataset and monitors the status of the generated data.
Outcome:	The proposed method improves safety and quality of three representative large language models against safety issues without sacrificing model utility.

Copied to clipboard

Emre Can Acikgoz, Cheng Qian, Heng Ji, Dilek Hakkani-Tür, Gokhan Tur

Green & Sustainable NLP

Challenge:	Existing methods for language model fine-tuning are expensive and inefficient . existing methods rarely assess whether a training sample provides novel information .
Approach:	They propose a test-time self-improvement algorithm that generates a sample that model struggles with . they also explore Test-Time Distillation, which leverages 'stronger supervisors'
Outcome:	The proposed algorithm improves performance with +5.48% absolute accuracy gain on average across benchmarks.

Emancipating Event Extraction from the Constraints of Long-Tailed Distribution Data Utilizing Large Language Models (2024.lrec-main)

Copied to clipboard

Zhigang Kan, Liwen Peng, Linbo Qiao, Dongsheng Li

Generating Data for Symbolic Language with Large Language Models (2023.emnlp-main)

Challenge:	Existing methods for EE depend on manual annotations, which are expensive and scarce.
Approach:	They propose to transform the event extraction task into multi-turn dialogues and a novel method for generating high-quality data.
Outcome:	The proposed methods significantly improve existing models’ performance with various paradigms and structures, especially on tail types.

Copied to clipboard

Jiacheng Ye, Chengzu Li, Lingpeng Kong, Tao Yu

What Has Been Lost with Synthetic Evaluation? (2025.findings-emnlp)

Challenge:	Large language models (LLMs) bring performance and complexity, but they incur a large computational cost in practice.
Approach:	They propose a task-based model which uses large language models to generate symbolic language data by an informative prompt and agreement-based verifier.
Outcome:	The proposed model can generate symbolic language data with a few human demonstrations and saves a considerable amount of inference effort.

Copied to clipboard

Alexander Gill, Abhilasha Ravichander, Ana Marasovic

Evolving Knowledge Distillation with Large Language Models and Active Learning (2024.lrec-main)

Challenge:	a recent study evaluated the validity and difficulty of large language models for evaluation benchmarks . large language model evaluation benchmarking is challenging and requires specific phenomena to be addressed .
Approach:	They compare LLM-generated reasoning-over-text benchmarks to those generated through crowdsourcing . they find they are less challenging for LLMs than their human-authored counterparts .
Outcome:	The results show that LLMs can generate variants that are valid according to annotation guidelines, but less challenging than human-authored counterparts.

Copied to clipboard

Chengyuan Liu, Fubang Zhao, Kun Kuang, Yangyang Kang, Zhuoren Jiang, Changlong Sun, Fei Wu

Leveraging QA Datasets to Improve Generative Data Augmentation (2022.emnlp-main)

Challenge:	Existing studies have focused on the direct use of large language models for text generation and labeling, without fully exploring their potential to comprehend the target task and acquire valuable knowledge.
Approach:	They propose to distill the knowledge of large language models into smaller models by generating annotated data.
Outcome:	The proposed method improves the performance of small domain models while enhancing the ability of large language models.

Copied to clipboard

Dheeraj Mekala, Tu Vu, Timo Schick, Jingbo Shang

Low-Resource NLP

Challenge:	Recent advances in NLP have substantially improved the capability of pretrained language models to generate high-quality text.
Approach:	They propose to reformulate data generation as context generation for a given question-answer (QA) pair and leverage QA datasets for training context generators.
Outcome:	The proposed approach improves performance for few-shot and zero-shot classification datasets on multiple classification dataset.

Generating Labeled Data for Relation Extraction: A Meta Learning Approach with Joint GPT-2 Training (2023.findings-acl)

Copied to clipboard

Amir Pouran Ben Veyseh, Franck Dernoncourt, Bonan Min, Thien Nguyen

Evaluating the Factual Consistency of Abstractive Text Summarization (2020.emnlp-main)

Challenge:	Relation Extraction (RE) is the task of identifying semantic relation between entities mentioned in text.
Approach:	They propose a framework to automatically generate labeled data for Relation Extraction . they propose 'reward function' to update pre-trained language model for RE .
Outcome:	The proposed framework generates labeled data for relation extraction using a pre-trained language model and a meta learning approach to improve the generated samples.

Copied to clipboard

Wojciech Kryscinski, Bryan McCann, Caiming Xiong, Richard Socher

Robustifying Sentiment Classification by Maximally Exploiting Few Counterfactuals (2022.emnlp-main)

Challenge:	a weakly-supervised approach is needed to verify factual consistency . auxiliary span extraction tasks are useful for verifying factual consistent summaries .
Approach:	They propose a weakly-supervised approach for verifying factual consistency . they transfer the model to summaries generated by several neural models .
Outcome:	The proposed approach outperforms models trained with strong supervision on source documents and human evaluations.

Copied to clipboard

Maarten De Raedt, Fréderic Godin, Chris Develder, Thomas Demeester

Robustness in NLP

Challenge:	a recent study found that finetuned language models rely on spurious patterns in training data . this limitation limits their performance on out-of-distribution (OOD) test data.
Approach:	They propose a method that only requires annotation of a small fraction of training data . they add 1% manual counterfactuals to training data and generate extra counterfacts in vector space .
Outcome:	The proposed approach improves sentiment classification using IMDb data and other sets for OOD tests.

INMT-Lite: Accelerating Low-Resource Language Data Collection via Offline Interactive Neural Machine Translation (2024.lrec-main)

Copied to clipboard

Harshita Diddee, Anurag Shukla, Tanuja Ganu, Vivek Seshadri, Sandipan Dandapat, Monojit Choudhury, Kalika Bali

Multilinguality

Challenge:	Interactive Neural Machine Translation (INMT) systems can be used to promote data collection in several under-resourced languages, but are often not adapted to the deployment constraints native language speakers operate in.
Approach:	They propose to use interactive neural machine translation systems to promote data collection in several under-resourced languages by integrating three different modes of Internet-independent deployment and four assistive interfaces suitable for data-sparse languages.
Outcome:	The proposed model improves the data generation experience of community members along multiple axes without compromising on the quality of the generated translations.

The State of Multilingual LLM Safety Research: From Measuring The Language Gap To Mitigating It (2025.emnlp-main)

Copied to clipboard

Zheng Xin Yong, Beyza Ermis, Marzieh Fadaee, Stephen Bach, Julia Kreutzer

Multilinguality

Challenge:	a systematic review of 300 publications reveals a language gap in LLM safety research . even high-resource non-English languages receive little attention, authors note .
Approach:	They propose to focus on safety evaluation, training data generation, and crosslingual safety generalization based on their findings.
Outcome:	The authors suggest that the field can develop more robust, inclusive safety practices for diverse global populations.

Ensemble-Instruct: Instruction Tuning Data Generation with a Heterogeneous Mixture of LMs (2023.findings-emnlp)

Copied to clipboard

Young-Suk Lee, Md Sultan, Yousef El-Kurdi, Tahira Naseem, Asim Munawar, Radu Florian, Salim Roukos, Ramón Astudillo

Knowledge-Infused Prompting: Assessing and Advancing Clinical Text Data Generation with Large Language Models (2024.findings-acl)

Challenge:	Empirical studies with different instruction-tuned LMs show that our proposed method yields higher-quality instruction tuning data than Self-Instruct.
Approach:	They propose to use in-context learning techniques to train strong conversational agents . they propose to categorize and simplify ICL templates to make prompt learning easier .
Outcome:	Empirical results show that the proposed method yields higher-quality instruction tuning data than Self-Instruct and improves performance of both vanilla and instruction-tuned LMs.

Copied to clipboard

Ran Xu, Hejie Cui, Yue Yu, Xuan Kan, Wenqi Shi, Yuchen Zhuang, May Dongmei Wang, Wei Jin, Joyce Ho, Carl Yang

Procedural Environment Generation for Tool-Use Agents (2025.emnlp-main)

Challenge:	Clinical natural language processing (NLP) is a subfield that requires the extraction, analysis, and interpretation of unstructured clinical text.
Approach:	They propose a model which infuses knowledge into clinical text generation with LLMs for clinical NLP tasks.
Outcome:	The proposed model improves performance across 8 clinical NLP tasks and 18 datasets by 7.7%-8.7% on average.

Copied to clipboard

Michael Sullivan, Mareike Hartmann, Alexander Koller

Challenge:	Existing approaches to curation of tool-use training data are non-interactive and/or non-compositional.
Approach:	They propose a pipeline for the procedural generation of interactive tools and compositional tool-use data.
Outcome:	The proposed pipeline improves on a range of tool-use benchmarks and sets the new SoTA for two metrics on the NESTFUL dataset.

Reverse Constitutional AI: A Framework for Controllable Toxic Data Generation via Probability-Clamped RLAIF (2026.findings-acl)

Copied to clipboard

Yuan Fang, Yiming Luo, Aimin Zhou, Fei Tan

Robustness in NLP

Challenge:	Existing approaches to red teaming focus on searching for individual adversarial inputs.
Approach:	They propose a framework for automated adversarial data generation that inverts harmless constitution into constitution of toxicity and iteratively refining model outputs through critique–revision pipeline.
Outcome:	The proposed framework generates diverse, high-quality toxic data without human annotation and significantly improves semantic coherence without sacrificing adversarial strength.

ToolGrad: Efficient Tool-use Dataset Generation with Textual “Gradients” (2026.findings-acl)

Copied to clipboard

Zhongyi Zhou, Kohei Uehara, Haoyu Zhang, Jingtao Zhou, Lin Gu, Ruofei Du, Zheng Xu, Tatsuya Harada

LoopTool: Closing the Data–Training Loop for Robust LLM Tool Calls (2026.acl-long)

Challenge:	Prior work synthesizes tool-use LLM datasets by first generating a user query, then complex tool-using annotations like DFS.
Approach:	They propose an agentic framework that synthesizes user queries and generates valid tool-use chains . they propose a dataset with more complex tool use, lower cost, and almost 100% pass rate .
Outcome:	Experiments show that tools trained on ToolGrad outperform expensive baseline datasets and proprietary LLMs.

Copied to clipboard

Kangning Zhang, Weiwen Liu, Wenxiang Jiao, Kounianhua Du, Yuan Lu, Weinan Zhang, Yong Yu

CmdCaliper: A Semantic-Aware Command-Line Embedding Model and Dataset for Security Research (2024.emnlp-main)

Challenge:	Large Language Models (LLMs) are powerful tools for multi-step tasks, but static data pipelines hinder tool learning and cause noisy labels to persist.
Approach:	They propose a fully automated, model-aware data evolution framework that tightly integrates data synthesis and model training.
Outcome:	Experiments show that LoopTool-8B significantly surpasses its 32B data generator and achieves new state-of-the-art results on the BFCL-v3 and ACEBench benchmarks for its scale.

Copied to clipboard

Sian-Yao Huang, Cheng-Lin Yang, Che-Yu Lin, Chun-Ying Huang

Prompting-based Synthetic Data Generation for Few-Shot Question Answering (2024.lrec-main)

Challenge:	Currently, command-line embeddings are limited due to the lack of comprehensive datasets for the field due to privacy and regulation concerns.
Approach:	They propose a command-line embedding model called CmdCaliper for training and unbiased evaluation using a set of large language models comprising 28,520 similar command- line pairs.
Outcome:	The proposed model suppresses state-of-the-art sentences with ten times more parameters across various tasks.

Copied to clipboard

Maximilian Schmidt, Andrea Bartezzaghi, Ngoc Thang Vu

Rule Discovery for Natural Language Inference Data Generation Using Out-of-Distribution Detection (2025.emnlp-main)

Challenge:	Language models have boosted the performance of Question Answering, but data annotation is costly.
Approach:	They propose to use large language models to improve Question Answering performance . they argue that domain-agnostic knowledge from LMs is sufficient to create a well-curated dataset.
Outcome:	The proposed model outperforms state-of-the-art approaches on few-shot Question Answering.

Copied to clipboard

Juyoung Han, Hyunsun Hwang, Changki Lee

Explainability & Interpretability in NLP

Challenge:	Existing training rules for natural language inference do not cover the diversity of natural language.
Approach:	They propose a framework that combines out-of-distribution detection and clustering to identify new premise–hypothesis pairs in a dataset that are not covered by existing rules.
Outcome:	The proposed framework achieves +0.85%p accuracy on 2k and +0.15%p on 550k samples.

Towards Inference-time Scaling for Continuous Space Reasoning (2026.findings-acl)

Copied to clipboard

Minghan Wang, Thuy-Trang Vu, Ehsan Shareghi, Gholamreza Haffari

Towards Algorithmic Fidelity: Mental Health Representation across Demographics in Synthetic vs. Human-generated Data (2024.lrec-main)

Challenge:	Recent advances in reasoning large language models have expanded along training and inferencetime dimensions.
Approach:	They propose to use COCONUT (CITATION) continuous space reasoning LM as the backbone to generate diverse reasoning paths through dropout-based sampling.
Outcome:	The proposed method could enable performance gains similar to those observed in the discrete space, but only marginally improves in the continuous space.

Copied to clipboard

Shinka Mori, Oana Ignat, Andrew Lee, Rada Mihalcea

Reasoning in a Combinatorial and Constrained World: Benchmarking LLMs on Natural-Language Combinatorial Optimization (2026.findings-acl)

Challenge:	Synthetic data generation has the potential to impact domains with scarce data, but we need to understand how different demographics are represented in it.
Approach:	They develop a procedure to generate depression data using GPT-3 and analyze it to uncover the types of stressors it assigns to demographic groups.
Outcome:	The proposed procedure produces depression data using GPT-3, and compares it to a human-generated dataset.

Copied to clipboard

Xia Jiang, Jing Chen, Cong Zhang, Jie Gao, Chengpeng Hu, Chenhao Zhang, Yaoxin Wu, Yingqian Zhang